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SUMMARY 


The  countries  of  the  world  are  undergoing  rapidly  changing  economic  and 
political  events.  In  addition,  the  United  States  defense  budget  continues  to  be  under 
pressure  to  provide  greater  efficiency  with  fewer  resources.  To  accomplish  this  the 
managers  of  large  defense  programs  must  use  innovative  and  effective  manage¬ 
ment  procedu^'es.  One  technique  that  will  assist  the  Program  Managers  in  this 
monumental  task  is  the  use  of  regimens  and  validated  protocols  to  plan  a  surviv¬ 
ability  validation  program.  The  program  management  team  can  select  the  approach 
best  suited  to  their  program,  in  order  to  provide  the  desired  confidence  in  the 
system  survivability  within  the  schedule  and  financial  constraints. 

This  paper  addresses  some  of  the  key  economic,  organizational,  and  institu¬ 
tional  issues  associated  with  the  development  and  use  of  protocols.  It  discusses 
factors  affecting  protocol  cost  (both  development  and  application  costs),  consider¬ 
ations  for  protocol  selection,  test-bed/simulator/ analysis  tool  availability,  organi¬ 
zational  and  instutional  issues  affecting  protocol  use,  deviations  precluding  adher¬ 
ence  to  validated  protocols,  and  protocol  advantages.  Developers  of  protocols 
should  be  aware  of  these  issues.  This  knowledge  will  assist  them  in  designing  more 
flexible  protocols  that  can  be  tailored  for  differing  circumstances  without  losing  the 
fidelity  or  assurance  that  using  the  protocol  will  produce  the  desired  survivability 
confidence  level. 

Using  validated  protocols  will  provide  extensive  savings  of  time  and  resources 
during  the  program  life-cycle,  primarily  due  to  the  use  of  pre-approved  systematic 
methods  to  obtain  the  prescribed  hardness  levels  and  associated  confidence 
required  for  \he  system  deployment.  However,  there  are  several  outstanding  issues 
that  will  be  considered  during  the  development  of  the  protocols.  These  are  listed 
below. 


(1)  An  adequate  list  of  commercial  and  government  test  facilities, 
standards,  handbooks,  data  items,  and  available  analysis/simuiation  codes  must  be 
updated  and  configuration  control  must  be  imposed  and  maintained  for  as  long  as 
the  database  is  utilized.  This  wii!  be  the  basis  set  for  the  protocols  and  will  be  used 
by  Program  Office  personnel  in  developing  the  survivability  regimen.  The  basis  set 
will  include  only  validated  protocols.  Thus,  community  acceptance  of  the  validation 
techniques  will  be  easier,  because  the  protocols  are  already  in  general  use; 

(2)  A  determination  must  be  made  as  to  the  level  of  generality  of  the 
protocols  required  to  make  a  viable  set  of  protocols  for  all  types  of  situations. 
Protocols  from  each  of  the  DoD  agencies  should  be  used  and  various  techniques 
specific  to  a  particular  agency  must  be  incorporated.  This  will  also  assist  in 
acceptance  of  the  protocols; 

(3)  The  resistance  of  agencies  against  adopting  a  new  management  tech¬ 
nique  must  be  addressed.  This  can  be  accomplished  by  informing  the  community 
through  conference  papers,  technical  meetings,  and  reports  addressing  the  advan¬ 
tages  of  using  protocols.  Support  for  the  new  management  technique  can  be 
obtained  by  inviting  the  interested  agencies  to  workshops  to  assist  in  developing 
some  of  the  protocols.  This  protocol  introduction  process  must  show  the  Program 
Managers  that  they  will  be  provided  with  enough  information  to  perform  efficient 
cost/confidence  trade-offs  during  protocol  selection.  In  additiotj,  it  must  be  shown 
how  the  protocols  will  help  the  Program  Office  with  hardness  assurance,  hardness 
validation,  and  hardness  maintenance/hardness  surveillance  (HM/HS). 


(4)  investigations  are  required  as  to  the  best  methods  available  to  validate 
the  survivability  of  a  system  that  cannot  be  tested,  such  as  a  system  that  has  widely 
separated  components.  Once  the  techniques  have  been  established,  the  protocols 
can  be  developed. 
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SECTION  1 


BACKGROUND 


This  paper  outlines  the  economic,  organizational,  and  institutional  factors 
involved  in  the  survivability  validation  process.  This  section  will  introduce  the  basic 
protocol  concept  and  the  Program  Office's  view  of  survivability.  The  remaining 
sections  will  discuss  protocol  cost  development  factors,  cost  considerations  for 
protocol  selection,  organizational  and  institutional  issues  affecting  protocol  use, 
deviations  precluding  adherence  to  validated  protocols,  and  protocol  advantages. 
The  last  section  will  provide  a  summary  and  list  of  suggested  tasks  to  assist  in 
protocol  development.  (This  section  utilizes  all  the  listed  references.  Where 
material  is  taken  directly  from  the  text  the  reference  is  annotated.) 

The  new  Integrated  Weapon  System  Management  concept  being  implemented 
by  the  various  DoD  agencies  empowers  a  single  manager  with  maximum  authority 
over  the  widest  range  of  weapon  system  program  decisions  and  resources.  Thus,  the 
Program  Manager  will  have  the  responsibility  for  the  system  during  its  entire  life- 
cycle.  Atypical  system  life-cycle  is  depicted  in  Figure  1-1.  The  implementation  of  this 
concept  will  provide  a  single  face  to  the  operational  commands  to  cover  all  aspects 
of  integrated  weapon  system  management  from  "cradle  to  grave,"  with  initiation 
no  later  than  Milestone  1  and  lasting  until  the  system  is  retired  or  canceled.  Some  of 
the  major  areas  that  Program  Managers  must  be  concerned  with  include; 

(1)  defining  the  system/facility  requirements  based  on  operational 
concepts  and  intended  usage; 

(2)  interfacing  between  program  management,  specification 
development,  design  development,  installation  development,  validation  testing, 
and  users  throughout  the  duration  of  the  program; 

(3)  identifying  hardness  elements,  how  they  function,  and  their  potential 
degradation  modes.  This  includes  the  interrelationship  of  hardness  with 
system/facility  operation,  configuration  control,  reliability,  and  maintainability; 

(4)  validating  the  survivability  of  the  system  through  a  combination  of 
tests,  simulations,  and  analyses; 

(5)  integrating  hardness  into  system/facility  design,  fabrication, 
acceptance,  operation,  and  maintenance; 

(6)  developing  requirements,  approaches,  and  procedures  for  maintaining 
hardness.  This  includes  the  role  of  hardness  maintenance  for  the  following  areas: 
periodic  inspections,  repair  activities,  repair  part  validation,  and  inclusion  in 
technical  manuals; 

(7)  utilizing  various  techniques/procedures  to  conduct  a  hardness 
surveillance  program  and  evaluate  hardness  element  performance  by  performing 
inspections  and  testing; 

(8)  identifying  support  and  test  equipment  to  perform  hardness  assurance, 
hardness  maintenance,  and  hardness  surveillance; 

(9)  providing  hardness  technical  data  for  system  operation/manuals; 

(10)  identifying  skills  and  manpower  required  for  hardness  assurance; 

(1 1)  defining  system/facility  maintenance  and  operation  to  provide 
hardness  assurance; 

(12)  developing  plans  for  hardness  maintenance  and  hardness  surveillance; 

(13)  instituting  personnel  training  in  hardness  awareness,  maintenance,  and 
surveillance. 


) 


» 


Figure  1*1  •  Acquisition  milestones  and  phases. 


To  assist  the  Program  Manager  in  adequately  performing  these  functions  for 
survivability,  an  innovative  manaqement  technique  is  required  to  streamline  the 
survivability  validation  process.  This  management  tool  is  effectively  provided  by  the 
use  of  protocols. 

A  survivability  validation  protocol  is  a  pre-defined,  ordered  set  of  tools  and 
procedures  that  must  be  applied  to  a  specihc  object  to  validate,  with  a  measurable 
statistical  confidence,  the  capability  to  perform  a  specified  mission  function  in  a 
defined  environment  resulting  from  a  specified  threat  class.  Application  of  the 
protocol  to  the  specific  object  produces  a  documented  collection  of  data  estab¬ 
lishing  audible  traceability  through  the  system  survivability  validation  process.  A 
regimen  is  defined  as  a  set  of  survivability  validation  protocols  required  to  assure  the 
specified  performance  of  the  system  for  its  validated  threat.  The  survivability 
regimen  would  be  defined  in  the  Test  and  Evaluation  Master  Plan  (TEMP)  by  the 
Program  Management  Office  and  would  be  approved  by  the  acquisition  authority  at 
Milestone  1 .  It  should  include  sufficient  detail  to  ensure  the  timely  availability  of 
both  existing  and  planned  test  resources  required  to  support  the  test  and  evaluation 
program.  The  TEMP  is  refined  throughout  the  acquisition  cycle  at  each  of  the  mile¬ 
stones  to  provide  increased  accuracy  for  the  survivability  validation  process  and 
assure  the  system  survivability.  This  necessitates  a  more  detailed  description  of  the 
tools  and  test  procedures  used  to  assure  survivability. 


SECTION  2 


FACTORS  AFFECTING  PROTOCOL  COSTS 


2.1  INTRODUCTION. 

To  be  an  effective  management  tool,  the  cost  of  developing  and  applying  a 
protocol  must  be  kept  to  a  minimum.  Any  new  validation  tools  required  to  perform 
the  protocol  must  be  considered  as  part  of  the  protocol  development.  Any  deficien¬ 
cies  in  the  protocols  or  the  validation  tools  must  be  identified  as  early  in  the  acqui¬ 
sition  cycle  as  possible.  Enough  lead  time  should  be  provided  for  the  development 
of  the  components  necessary  for  the  protocol.  If  this  is  not  done,  then  the  hardness 
validation  process  will  become  the  critical  path  in  the  system  development.  In  some 
cases,  the  required  protocol  components  may  require  a  longer  development  time 
than  the  system  being  designed  in  the  acquisition  cycle.  For  example:  if  the  mission 
requirements  for  a  system  indicate  that  underground  testing  (UGT)  is  needed,  but  a 
ban  is  imposed  on  underground  testing,  then  a  simulator  might  be  developed  to  fill 
the  need.  The  acquisition  process  for  the  simulator  may  take  as  long  or  longer  than 
is  allocated  in  the  schedule  of  the  system  requiring  the  testing.  For  this  case,  other 
analyses  or  tests  that  provide  a  reasonable  hardness  confidence  level  would  need  to 
be  performed  until  the  simulator  is  available. 

Initially,  there  will  be  some  nonrecurring  costs  incurred  in  the  development  of 
the  protocol  formats  and  establishment  of  the  databases.  But  as  the  protocols  are 
utilized,  only  the  cost  of  developing  a  new  protocol  and  insertion  of  it  into  the 
database  will  be  incurred.  However,  some  of  the  factors  impacting  the  development 
and  application  costs  of  a  new  protocol  are  listed  below  and  will  be  addressed  in  the 
following  paragraphs: 

(1)  complexity  of  system  to  be  hardened; 

(2)  level  of  detail  of  analysis  or  test; 

(3)  magnitude  of  threat  and  deployment  environment; 

(4)  complexity/sophistication  of  protocols; 

(5)  database  development  and  configuration  management  of  applicable 
standards  and  specifications; 

(6)  survivability  confidence  limits; 

(7)  cost  of  producing  and  validating  needed  software  analysis  tools; 

(8)  protocol  application  costs. 

2.2  COMPLEXITY  OF  SYSTEM  TO  BE  HARDENED. 

The  protocols  in  a  regimen  will  address  the  different  levels  of  integration  from 
the  highest  level,  a  "system-of-systems"  (SOS),  to  the  lowest  levels  incorporating 
piece-parts  and  materials  (refer  to  Table  7-2).  The  protocols  can  range  from  simple 
instructions,  such  as  what  military  standard  can  be  used  to  validate  the  survivability 
of  a  component,  to  a  detailed  end-to-end  simulation  of  system-of-systems,  such  as  a 
GPALS.  A  protocol  can  refer  to  testing,  analysis,  simulation,  or  a  combination  of 
these  techniques  to  perform  the  survivability  validation.  As  the  system  becomes 
more  complex,  the  more  detailed  the  protocol  must  become  so  as  to  be  confident  of 
the  system  survivability. 

The  protocols  for  the  lowest  level  in  Table  7-2,  the  part/material  level,  have  been 
under  development  for  many  years  and  are  fairly  well  defined.  As  new  parts  are 
developed,  better  processes  are  discovered  to  produce  existing  parts,  or  threats 


change,  new  protocols  may  be  required  to  address  these  areas.  The  protocol 
development  to  evaluate  the  hardness  of  piece-parts  would  typically  require  minor 
modifications  to  the  existing  protocols  in  the  database. 

As  the  complexity  or  size  of  the  system  increases,  the  modifications  to  existing 
protocols  become  less  straightforward  and  may  require  the  development  of  entirely 
new  protocols.  Unless  the  the  system  is  utilizing  a  totally  new  technology,  enhance¬ 
ments  to  available  protocols  can  be  made  as  mentioned  earlier  for  the  piece-parts. 

The  regimen  and  protocols  for  system-of-systems  could  be  built  upon  or  include 
the  protocols  utilized  for  the  lower  level  systems  in  the  hierarchy  of  levels  (see  Figure 
8-2).  However,  the  regimen  must  contain  the  procedures  to  evaluate  the  inter¬ 
connection  of  the  components  and  the  composite  system  as  a  whole.  If  the  system 
requires  an  entirely  new  protocol,  the  requirement  for  the  protocol  development 
should  be  identified  as  early  in  the  procurement  cycle  as  possible. 

2.3  LEVEL  OF  DETAIL  OF  ANALYSIS  OR  TEST. 

The  level  of  detail  of  the  analysis  or  test  must  be  considered  in  the  development  of  a 
new  protocol.  For  example,  a  protocol  referring  to  a  test  method  in  a  military 
standard  will  require  less  development  time  to  implement  than  a  detailed  mathe¬ 
matical  calculation  using  the  latest  method-of-moments  techniques  for  coupling  of 
an  electromagnetic  wave  to  a  cavity  of  arbitrary  shape.  Validation  of  the  surviv¬ 
ability  can  be  performed  by  many  methods  to  include  similarity  of  the  compop.er>ts 
to  other  previously  tested  components  or  use  of  previously  tested  components, 
actual  testing  at  expected  threat  levels,  analysis  using  estimated  threat  levels,  and 
simulation  of  the  threat  and/or  system  components.  The  method  selected  is 
dependent  on  the  survivability  confidence  level  required  by  the  acquisition 
authority. 

2.4  MAGNITUDE  OF  THE  THREAT  AND  DEPLOYMENT  ENVIRONMENT. 

The  threat  to  be  countered  and  the  projected  deployment  environment  will  be 
defined  during  the  process  of  identifying  the  mission  needs  or  deficiencies.  In 
addition,  a  system  threat  assessment  will  be  performed  prior  to  each  milestone 
decision  point.  The  assessment  will  address  projected  capabilities  that  a  potential 
enemy  could  use  to  defeat,  destroy,  degrade,  or  deny  the  effectiveness  of  a  pro¬ 
posed  concept  or  system  being  developed  or  produced.  The  System  Threat  Assess¬ 
ment  Report  (STAR)  is  prepared  by  the  various  DoD  components  and  it  contains  the 
Defense  Intelligence  Agency  validated  threats.  The  STAR  is  the  basic  authoritative 
system  threat  assessment  tailored  for  and  focused  on  a  particular  defense  acquisition 
program.  In  addition,  the  STAR  is  updated  during  each  acquisition  phase  as 
determined  by  the  milestone  decision  authority. 

The  types  of  threats  (nuclear  effects,  kinetic  energy  weapon  effects,  directed 
energy  weapon  effects,  or  other  electromagnetic  weapon  effects)  and  the  scenario 
will  determine  the  complexity  and  hence  the  cost  of  the  protocols  developed.  For 
example,  systems  to  be  used  in  a  tactical  nuclear  environment  may  not  require  the 
same  level  of  protection  or  survivability  confidence  levels  as  systems  used  in  a 
strategic  nuclear  environment.  Thus,  the  protocols  to  validate  survivability  for  each 
type  of  environment  may  have  some  similarities,  but  they  may  differ  in  the  level  of 
hardness  or  confidence  level  required  for  the  system  to  survive  in  the  anticipated 
environment.  Likewise,  the  number  of  threats  to  be  countered  by  a  system  will 
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determine  the  complexity  and  number  of  the  protocols  in  the  regimen  to  validate 
survivability  of  the  system. 

The  protocols  for  air,  land,  sea,  and  space  environments  can  have  some 
similarities,  but  each  will  have  requirements  peculiar  to  that  environment.  Thus  the  ’ 

protocols  for  systems  to  be  deployed  in  a  specific  environment  must  be  tailored  to 
meet  the  survivability  assurance  levels  of  that  environment. 

2.5  COMPLEXITY/SOPHISTICATION  OF  PROTOCOLS. 

As  mentioned  earlier,  a  protocol  can  be  as  simple  as  referring  to  a  specific  test  * 

procedure  to  validate  the  hardness  of  a  component  or  it  can  be  a  complex  simulation 
code  to  mimic  a  nuclear  engagement.  As  the  use  of  sophisticated  software/ 
hardware  simulations  increases,  the  modeling  of  actual  events  becomes  more 
realistic.  These  methods  can  be  used  to  augment  actual  testing  or  permit  the  testing 
of  components  to  be  waived  during  the  validation  process,  depending  upon  the 
level  OT  confidence  in  the  hardening  predictions.  I 

The  level  of  sophistication  of  software  can  be  increased  by  incorporating 
artificial  intelligence  techniques.  Expert  systems  can  be  used  to  enhance  the  decision 
making  capabilities  of  a  program.  By  incorporating  the  knowledge  of  "expert 
individuals"  into  a  rule  base  used  by  the  program,  the  resulting  decisions  incor¬ 
porate  the  human  expertise.  Neural  networks  can  be  used  to  predict  future  trends  > 

based  on  past  "learned"  data.  The  result  will  be  an  approximation  of  the  actual 
answer.  For  predictions  dealinq  with  uncertain  events  or  data,  fuzzy  logic  can  be 
utilized.  Fuzzy  logic  can  be  embedded  into  other  applications  to  provicie  more 
accurate  predictions. 

2.6  DATABASE  DEVELOPMENT  AND  CONFIGURATION  MANAGEMENT  OF  , 

APPLICABLE  STANDARDS  AND  SPECIFICATIONS. 

As  the  regimens  and  protocols  are  developed,  a  database  will  be  built  of  existing 
protocols,  test  methods,  analysis  tools,  simulators,  test  facilities,  specifications,  etc. 

Initially,  an  expenditure  of  time  and  resources  will  be  required  to  gather  adequate 
documentation  on  these  items.  The  database  will  only  need  to  be  updated  as  new  ^ 

protocols  are  developed.  Configuration  management  of  the  database  is  an  on¬ 
going  process  with  its  associated  expenses,  and  must  be  continued  as  long  as  the 
data  base  is  maintained.  Program  Office  personnel  will  then  have  the  opportunity 
to  select  from  this  array  of  tools  to  build  an  acceptable  survivability  program  which 
will  address  the  required  confidence  limits.  If  the  necessary  tools  are  not  available, 
then  they  must  be  developed  by  the  time  they  are  needed.  Otherwise,  the  program 
schedule  can  be  impacted  and  the  survivability  validation  will  become  the  critical  ' 

path  in  the  program  schedule. 

2.7  SURVIVABILITY  CONFIDENCE  LIMITS. 

The  type  of  protocol  selected  for  use  will  depend  on  the  confidence  limits 
required  tor  the  system  survivability.  Sufficient  testing  of  the  system  in  the  expected  ^ 

environment  with  the  anticipated  threats  can  assure  tne  system  survivability. 

Needless  to  say,  actual  testing  against  most  threats  cannot  be  performed  due  to  the 

difficulty  of  the  test  or  the  severity  of  the  testing  to  the  environment,  such  as  in 

nuclear  explosions.  Thus  analyses,  simulations,  preliminary  testing,  or  detailed 

testing  (such  as  UGTs)  must  be  used.  If  the  system  can  undergo  an  underground  test, 

the  results  will  usually  be  the  most  realistic,  although  even  a'UGT  will  not  totally  > 
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emulate  an  atmospheric  event,  e.g.,  radiation  pulse  width,  x-ray  spectrum,  etc.  The 
protocols  to  perform  a  U6T  may  be  relatively  simple,  but  the  actual  performance  of 
the  test  is  one  of  the  most  expensive  means  of  survivability  validation.  Since  nuclear 
testinq  is  infrequent  and  expensive,  other  methods  can  be  used  to  provide  basic 
confidence  in  the  survivability.  The  fidelity  of  the  other  methods  can  be  refined  by 
comparison  with  nuclear  test  data.  Large  system-of-systems  testing  can  only  be 
performed  on  smaller  segments,  so  survivability  validation  of  the  entire  ensemble 
must  be  performed  by  analysis  or  simulation.  The  greater  the  confidence  in  the 
result,  the  more  extensive  the  protocol  validation  becomes  and  the  greater  the 
development  cost. 

2.8  COST  OF  PRODUCING  AND  VALIDATING  NEEDED  SOFTWARE  ANALYSIS  TOOLS. 

Protocols  can  utilize  various  computer  and  analytic  simulations  as  part  of  their 
structure.  Many  times  new  modules  can  be  inserted  into  the  old  codes  to  provide 
enhancements  to  validate  the  survivability  of  the  new  component.  Otherwise,  a 
new  code  will  need  to  be  developed  to  perform  the  required  functions.  In  addition, 
the  software  utilized  must  be  validated  to  assure  its  usefulness  in  providing  suscepti¬ 
bility  validation.  Independent  of  the  method  that  is  used,  time  and  resources  must 
be  allotted  in  the  acquisition  cycle  to  develop  and  validate  the  new  software 
enhancements  to  the  required  confidence  levels. 

2.9  PROTOCOL  APPLICATION  COSTS. 

Once  the  regimen  has  been  built  with  protocols  selected  and  adapted  for 
system-specific  use  by  the  Program  Manager  and  his  staff,  the  cost  of  applying  the 
protocols  must  be  considered.  The  Program  Manager  must  weigh  the  impact  of 
continuinq  his  organization's  present  mode  of  operation  versus  the  impact  of  imple¬ 
menting  the  protocols  outlined  in  the  regimen.  Although  the  present  mode  of  sur¬ 
vivability  validation  operations  performed  by  many  organizations  may  have  pro¬ 
duced  adequate  results  in  the  past  and  may  be  favorecT because  of  familiarity  with 
the  procedures,  their  continued  use  may  be  time-consuming  and  a  waste  of 
resources.  This  is  because  the  old  methods  have  evolved  over  many  years,  and  due  to 
resistance  to  change,  may  not  use  the  latest  techniques  nor  be  easily  tailored  to 
specific  requirements.  These  factors  may  cause  wasted  effort  and  the  possibility  of 
cost  overruns,  schedule  slips,  or  waivers. 

The  use  of  protocols,  which  utilizes  a  set  of  quantifiable  processes,  alleviates  the 
undesirable  results  of  the  previous  factors.  This  is  because  the  protocol  baseline  will 
consist  of  relevant  survivability  validation  and  design  procedures.  They  will  have 
been  adequately  validated,  incorporate  lessons  learned,  and  be  able  to  be  tailored 
for  specific  requirements.  The  protocol  approach  will  permit  long  term  planning, 
since  the  costs  and  schedule  can  be  easily  quantified  from  the  beginning  of  the 
program  to  its  projected  completion. 

Recurring  cost  from  program  to  program  will  be  reduced,  because  the  protocol 
concept  will  provide  for  a  dynamic  database  of  validated  protocols.  This  will  elimi¬ 
nate  "reinventing  the  wheel"  for  each  new  program.  As  new  protocols  are  vali¬ 
dated,  they  will  be  added  to  the  database,  providing  the  Program  Managers  with 
current  validation  tools  and  lessons  learned  from  previous  programs.  Any  validation 
tools  needed  for  the  program  will  be  able  to  be  quickly  identified  to  permit  rapid 
projections  of  the  development  costs  of  the  tool  into  the  acquisition  process.  If  the 
tool  is  relatively  inexpensive  (such  as  a  short  subroutine),  the  Program  Office  may  be 
able  to  absorb  the  development  cost.  However,  if  the  tool  requires  significant 


development  cost  (such  as  a  simulation  facility),  a  new  program  may  be  initiated  to 
develop  it.  Either  way,  development  costs  will  be  reduced  due  to  the  early 
identification  of  the  needed  tools  in  the  planning  process. 
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SECTION  3 


ORGANIZATIONAL  AND  INSTITUTIONAL  ISSUES  AFFECTING  PROTOCOL  USE 

3.1  INTRODUaiON. 

There  are  several  factors  affecting  the  decisions  to  use  protocols.  Since  they 
cover  various  topic  areas,  they  will  be  considered  under  the  heading  of  "organi¬ 
zational  and  institutional"  issues.  The  following  sections  will  discuss  a  few  of  these 
organizational  and  institutional  issues. 

3.2  ORGANIZATIONAL  RESISTANCE  TO  ADOPTING  PROTOCOLS. 

It  is  "human  nature"  to  be  cautious  or  reluctant  to  adopt  new  ideas  or  to  change 
established  procedures.  Some  of  this  can  be  related  to  the  inertia  of  always  having 
done  something  a  certain  way  and  not  wanting  to  try  new  methods.  However,  in 
the  case  of  accepting  the  use  of  protocols,  there  are  several  other  factors  that  come 
Into  play.  Some  of  the  organizations  have  a  "not  developed  here"  attitude  and  will 
not  utilize  products  produced  by  other  organizations.  This  attitude  can  develop 
because  of  a  distrust  of  the  other  organization’s  understanding  of  the  problem, 
based  on  past  "bad"  experiences.  Sometimes,  it  is  due  to  the  lack  of  control  of  the 
people  doing  the  work  and  little  control  on  the  resulting  product.  Another  reason  is 
a  protective  attitude  where  the  manager  wants  to  keep  the  funds  for  the  project 
within  his  own  organization  and  keep  his  people  busy,  making  his  department  look 
productive.  Unfortunately,  this  process  can  be  repeated  in  several  agencies,  so  that 
there  is  a  duplication  of  similar  efforts.  The  reduced  budgets  projected  for  future 
years  will  not  allow  this  type  of  management  to  continue.  Thus  organizations  will 
be  forced  to  tradeoff  the  merits  of  continuing  with  their  present  means  of  validation 
versus  using  the  protocol  management  process. 

The  various  DoD  components  have  somewhat  different  requirements  for 
systems  developed  for  land,  sea,  air,  and  space.  However,  there  are  many  common 
methodology  elements  that  exist  between  the  systems  and  these  have  precipitated 
some  of  the  duplication  in  the  development  methodologies.  By  incorporating  all  the 
validated  hardening  techniques  into  a  database,  the  Program  Managers  in  each 
agency  can  select  the  best  techniques  for  their  system. 

3.3  EFFECT  OF  BUDGET  FLUCTUATIONS. 

Budget  fluctuations  can  have  a  serious  impact  on  the  use  of  protocols.  If  the 
world  situation  is  stabilized,  the  budget  will  reflect  cutbacks  in  defense.  The  impor¬ 
tance  of  the  project  to  the  national  (Jefense  will  determine  the  amount  of  funding  it 
receives.  If  the  anticipated  funding  level  is  not  provided  for  the  project;  the 
program  can  be  stretched  out  over  a  longer  time  period,  various  segments  of  the 
project  can  be  delayed  until  later  in  the  program,  protocols  producing  a  lower 
hardness  confidence  level  may  need  to  be  used  to  keep  within  the  budget,  or  the 
program  or  parts  of  it  could  be  canceled.  A  de-emphasis  on  the  project  reflected  by 
a  reduction  of  funding  will  necessitate  appropriate  changes  in  the  protocols  used  for 
the  program.  If  aggression  by  a  country  on  its  neighbors  is  in  process  or  being 
threatened,  the  defense  budget  will  reflect  the  severity  of  the  situation  and  the 
program  can  be  accelerated.  In  such  a  case,  more  efficient  (but  probably  more 
costly)  protocols  will  need  to  be  considered  to  meet  the  schedule  constraints.  Any  of 
these  options  will  require  a  reevaluation  of  the  protocols  and  reviews/approvals  by 
the  acquisition  authority. 


3.4  DIFFICULTY  IN  UTILIZING  FACILITIES. 


► 


Simulation  or  test  facilities  can  be  difficult  to  use  as  validation  tools  because  of  , 

the  problems  involved  in  fully  utilizing  them.  Some  of  these  problems  can  arise  from 
the  reluctance  of  agencies  to  use  other  facilities.  As  mentioned  earlier,  this  can  stem 
from  a  "not  developed  here"  attitude  or  a  sense  of  "loss  of  confo!  "for  the  testing 
by  permitting  other  agency  involvement  in  the  tests.  Other  situations  impacting 
utilization  include:  domination  of  the  facility  by  the  operating  agency  to  test  their 
own  systems,  unavailability  of  facility  due  to  heavy  test  schedule  or  repairs,  bumping 
of  projects  due  to  other  high  priority  projects,  or  the  required  capability  has  not 
been  implemented  by  the  time  the  program  needs  it.  These  types  of  problems  can 
be  devastating  to  the  system  test  schedule  even  to  the  extreme  of  missing  a  mile¬ 
stone.  Thus,  tne  Program  Office  must  have  contingency  plans  available.  The 
protocol  database  could  assist  in  this  process  because  it  will  have  all  the  options 
readily  available  for  review.  ^ 

3.5  THREAT  INTERPRETATION. 

Although  the  threat  environments  are  defined  in  the  STAR,  applicability  of  each 
of  the  components  in  a  system  to  those  threat  environments  must  be  assessed.  This 
is  true  particularly  for  distributed  systems  having  land,  sea,  air,  and  space  sub¬ 
systems.  These  system?  will  require  procedures  that  are  different  for  the  various  * 

components  in  tne  system. 

Infrequently,  the  intelligence  agencies  may  not  agree  on  the  interpretation  of 
the  intelligence  data  alluding  to  the  threat.  This  can  lead  to  heated  discussions 
between  agencies  with  the  result  being  that  the  threat  scenario  may  have  a  com¬ 
ponent  missing  from  it.  If  that  missing  component  is  later  deemed  a  valid  threat  or  if  ► 

a  new  threat  was  not  identified  in  the  original  STAR,  then  the  protocols  will  need  to 
be  modified  or  new  ones  developed  to  assess  the  hardness  of  the  system  against  the 
new  threat.  The  primary  danger  of  this  process  is  that  survivability  technology 
requirements  may  be  missed,  which  will  diminish  the  appropriate  lead-time  needed 
for  the  technology  development.  This  can  cause  schedule  slippages  or  the  inability 
to  assure  the  survivability  of  the  system  to  the  threat.  ^ 
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SE£T?ON  4 

CONSIDERATIONS  FOR  SELECTION  OF  TYPE  OF  PROTOCOL 

4.1  INTRODUCTION. 

The  survivability  validation  regimen  and  protocols  used  for  a  specific  system 
development  will  be  updated  as  the  acquisition  process  progresses  through  the 
various  milestones  and  phases.  The  regimen  and  protocols  may  be  modified  in 
conjunction  with  the  modifications  to  the  TEMP.  The  protocols  may  become  more 
comprehensive  with  each  phase  until  the  system  is  deployed.  For  example,  a 
protocol  developed  for  Phase  III  (Production  and  Deployment)  and  Phase  IV 
(Operations  and  Support)  will  stress  the  hardness  maintenance  and  hardness 
surveillance  of  the  operational  system  to  assess  its  contiiiued  survivability.  Typically, 
more  detailed  protocols  will  be  used  during  the  validation  testing  of  the  system  to 
meet  the  survivability  requirements.  The  confidence  level  of  the  protocol  will  be 
determined  by  the  system  requirements  and  the  extent  of  validation  of  the  protocol. 
Hopefully,  all  the  protocols  will  be  available  and  validated  prior  to  the  time  they  are 
required  for  the  program.  The  Program  Management  Team  will  select  the  appro¬ 
priate  validated  protocol  commensurate  with  the  TEMP  requirements  and  program 
financial/schedule  constraints.  Several  of  the  factors  the  program  management 
personnel  must  consider  in  selecting  protocols  are  listed  below  arid  described  in  the 
following  paragraphs: 

(‘t  )  availability  of  technology; 

(2)  schedule; 

(3)  size/complexity  of  system; 

(4)  fidelity  of  validation  and  confidence  limits  desired; 

(5)  type  of  deployment  environment; 

(6)  fiscal  constraints  imposed  on  program; 

(7)  availability  of  validation  tools; 

(8)  system  criticality. 

4.2  AVAILABILITY  OF  TECHNOLOGY. 

Updated  or  new  protocols  may  be  needed  if  a  new  technology  is  used  in  the 
production  of  the  system  under  development  or  new  technologies  are  utilized  in 
performing  old  processes.  Protocols  using  existing  simulators  or  analysis  tools  may 
be  able  to  be  upgraded  to  accommodate  the  new  technology  survivability  validation 
requirements.  The  technology  may  not  be  available  to  develop  the  necessary 
protocols  or  modifications.  Therefore,  the  need  for  protocol  development  should  be 
identified  as  soon  as  possible  in  the  acquisition  cycle.  If  the  protocol  requires  the 
development  of  a  new  capability  that  will  take  longer  than  the  projected  program 
survivability  validation  schedule,  other  means  must  used  to  provide  the  survivability 
validation.  These  can  consist  of  utilizing  other  less  effective  test  methods,  software 
simulations,  or  various  analyses  to  provide  a  level  of  confidence  of  the  system 
hardness.  The  lower  confidence  limits  available  from  these  methods  must  be 
weighed  against  the  impact  to  the  program  of  waiting  to  use  the  newly  developed 
facility. 

4.3  SCHEDULE. 

Protocol  selection  is  affected  by  the  program  schedule.  Trade-offs  between  the 
use  of  simulators,  analyses,  or  test  facilities  must  be  made  to  accommodate  the 
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program's  schedule.  Some  of  the  factors  to  be  weighed  in  this  decision  are:  time 
involved  to  conduct  tests,  time  required  to  develop  or  modify  test  facilities  or 
analysis  tools,  availability  and  cost  of  a  facility,  he  w  many  facilities  must  be  utilized 
in  the  validation  process,  availability  and  cosi  of  analysis  tools  or  simulators,  types  of 
analysis  tools  or  simulators  needed,  ease  of  obtaining  analysis  tools  or  simulators, 
and  confidence  in  the  obtained  hardness  levels.  Depending  upon  the  system  com¬ 
plexity,  many  times  a  combination  of  analyses/simulations  and  testing  provide  the 
best  compromise  between  survivability  validation  time  and  cost  factors. 

4.4  SIZE/COMPLEXITY  OF  SYSTEM. 

The  size  and  complexity  of  the  system  determines  the  methods  to  be  used  for 
survivability  validation.  An  expensive  test  facility  should  not  be  used  to  evaluate  a 
simple  system  that  can  be  analyzed  just  as  effectively  with  an  inexpensive  computer 
analysis  code.  Alternatively,  a  simulation  or  analysis  method  may  need  to  be  used  to 
avoid  costly  modifications  to  a  test  facility  to  test  an  oversized  system.  In  many  cases 
the  survivability  of  an  oversized  system  or  one  that  has  components  separated  by  a 
large  distance  can  only  be  validated  by  simulations  or  analysis  methods. 

4.5  FIDELITY  OF  VALIDATION  AND  CONFIDENCE  LIMITS  DESIRED. 

The  degree  of  hardness  validation  required  will  determine  the  fidelity  of  the 
validation  tool.  In  addition,  increasing  the  hardness  confidence  bound  will  usually 
increase  the  cost  of  implementation  of  the  protocol.  For  instance,  the  greater  the 
confidence  desired  from  a  software  program,  the  more  fidelity  the  program  requires 
(this  typically  is  manifested  in  more  lines  of  code  or  code  testing  with  their  assoc¬ 
iated  costs).  Higher  confidence  levels  often  imply  more  tests,  accompanied  by 
increased  costs  and  impacts  on  the  schedule. 

4.6  TYPE  OF  DEPLOYMENT  ENVIRONMENT. 

The  deployment  environment,  including  the  threats  and  operational  scenario,  is 
defined  in  the  mission  need  determination  and  initial  concept  studies.  To  optimize 
program  resources,  protocols  should  not  be  selected  that  will  validate  more  surviv¬ 
ability  areas  than  are  identified  in  the  TEMP  requirements.  However,  the  associated 
validation  protocols  should  be  tailored  to  provide  optimum  use  of  the  method.  For 
example,  as  mentioned  earlier,  it  is  already  recognized  within  the  DoD  that  some 
electromagnetic  codes  used  for  predicting  electromagnetic  compatibility  can  be 
used  to  assist  in  predicting  nuclear  effects.  Instead  of  using  the  full  capability  of  a 
code  to  predict  intra-system  compatibility,  intar-modulation,  cross-modulation  and 
radiated  effects,  the  code  can  be  tailored  to  use  only  the  radiation  effects  for  EMP 
predictions. 

4.7  FISCAL  CONSTRAINTS  IMPOSED  ON  PROGRAM. 

Monetary  restraints  on  the  program  can  be  a  key  driver  in  protocol  selection. 
This  is  because  the  Program  Manager  must  be  prudent  in  his  selection  of  protocols  to 
permit  the  maximum  efficiency  of  survivability  confidence  with  the  minimum 
expenditure  ot  resources.  This  means  that  if  there  are  two  protocols  that  can 
perform  a  validation  function  and  one  has  a  hardness  confidence  limit  5%  greater 
than  the  other,  but  it  costs  twice  as  much  to  perform,  the  tendency  will  be  to  use  the 
least  expensive  method  as  long  as  the  hardness  confidence  limit  is  acceptable. 
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The  Program  Manager  must  be  aware  of  fiscal  constraints  in  the  protocol 
selection  process.  Although  there  is  some  flexibility  in  the  use  of  the  money  allo¬ 
cated  for  the  program,  the  Program  Manager  must  make  sure  not  to  exceed  the  i 

amount  projected  for  the  tasks  and  he  must  stay  within  the  proposed  expenditure 
rate  as  much  as  possible.  As  a  result,  a  protocol  that  can  be  performed  within  the 
monetary  and  schedule  constraints  will  be  preferred  over  a  protocol  that  will  extend 
the  schedule  or  exceed  the  funds  allotted  for  the  task.  This  is  because  the  budgetary 
process  cannot  always  be  depended  upon  to  provide  additional  funding  in  subse¬ 
quent  years.  I 

4.8  AVAIUBILITY  OF  VALIDATION  TOOLS. 

Once  a  protocol  is  selected,  the  availability  of  the  tool  to  perform  the  surviv¬ 
ability  validation  must  be  commensurate  with  the  program  scnedule.  If  the  tools  are 
not  available,  then  the  need  for  a  development  program  must  be  identified.  If  the 
tools  cannot  be  produced  within  the  reouired  timeframe,  either  alternate  methods  * 

of  validation  must  be  found,  the  survivability  validation  requirement  must  be 
relaxed,  another  protocol  may  be  considereo  for  use,  or  a  program  slip  may  occur. 

Program  slips  with  their  corresponding  cost  overruns  are  not  looked  upon  favorably 

by  the  acquisition  authority  and  must  be  avoided.  Section  4  discusses  more  of  the 

issues  affecting  validation  tool  availability.  ^ 

4.9  SYSTEM  CRITICALITY. 

The  importance  of  the  system  to  the  national  defense  will  determine  the  level  of 
support  provided  by  upper  level  management.  Top  level  management  support  can 
also  expedite  the  scheduling  of  facilities  and  simulators.  However,  expedited 
schedules  usually  mean  elevated  implementation  costs.  i 

4.10  TEST-BED/SIMULATOR/ANALYSIS  TOOL  AVAILABILITY. 

The  system  being  developed  by  a  Program  Office  must  have  its  survivability 
validated  to  a  specified  hardness  level.  This  is  done  by  utilizing  validated  analysis 
techniques  to  predict  the  system  hardness  levels,  running  established  simulations  to  > 

provide  assurance  of  the  system  survivability,  or  performing  AGT  or  UGT  testing  on 
the  system  using  proven  test  methods.  The  availability  of  these  tools  is  imperative  to 
the  success  of  the  survivability  validation  program.  As  mentioned  befo^^e,  if  the 
various  tools  necessary  to  peitorm  the  validation  are  not  sufficient  to  perform  the 
task  or  they  are  not  in  existence,  a  development  program  for  these  validation  tools 
must  be  initiated.  As  a  result,  all  the  factors  mentioned  earlier  impacting  the  > 

development  of  a  new  protocol  must  be  considered. 

The  Program  Manager  must  evaluate  the  validation  tools  available  to  perform  a 
specific  protocol.  Some  of  the  factors  that  must  be  considered  include;  the  suffi¬ 
ciency  of  the  validation  tool  capabilities,  prediction  fidelity,  scheduling  of  validation 
facilities,  cost  of  validation  tool  use,  and  contingencies.  ^ 

The  validation  tools  must  have  capabilities  that  are  consistent  with  the  system  to 
be  analyzed.  The  tools  must  provide  the  proper  level  of  confidence  in  the  hardness 
of  the  system  to  assure  its  survivability  in  the  projected  threat  environment.  In 
addition,  the  fidelity  of  the  tools  must  be  evaluated  to  ensure  that  the  tools  provide 
hardness  assurance  to  the  proper  confidence  level.  A  confidence  level  excessively 
higher  than  that  required  for  the  system  is  generally  indicative  of  increased  costs  ' 

associated  with  the  use  of  the  valiclation  tool.  For  example,  scheduling  an  under- 
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ground  test  for  piece-parts  that  already  have  a  wealth  of  data  accumulated  on  them 
would  be  a  waste  of  program  resources.  Even  though  the  UGT  would  provide  the 
required  confidence  levels,  the  same  level  of  confidence  could  be  provided  by  AGTs 
or  analyses  and  there  would  be  substantial  reductions  in  difficulty  and  cost. 
Conversely,  analysis  of  a  new  component  using  a  technology  never  tested  before,  or 
having  suspected  modes  of  failure  different  from  what  can  be  induced  by  AGT, 
would  probably  not  provide  the  required  confidence  levels,  and  UGT  testing  may  be 
required  (depending  upon  the  importance  of  the  system). 

In  this  evaluation  process,  the  Program  Manager  must  consider  contingencies  if 
the  validation  tool  is  not  available.  Otner  options  must  be  considered  in  case  a 
simulator  or  test  facility  is  not  available  for  use  at  the  required  time  in  the  program 
schedule.  There  may  be  scheduling  difficulties  because  of  excessive  use  of  a  facility, 
other  higher  priority  systems  having  precedence  and  bumping  previously  scheduled 
programs,  reluctance  of  organizations  to  use  outside  facilities,  difficulty  in  coordi¬ 
nating  between  DoD  components,  changes  in  U  S  policy  regarding  nuclear  testing, 
etc.  For  example,  an  acquisition  program  may  have  several  key  subsystems  sched¬ 
uled  for  UGT  testing  when  the  United  States  initiates  a  ban  on  all  nuclear  testinq. 
The  protocol  the  Program  Manager  used  for  the  survivability  validation  should  have 
indicated  alternatives  to  UGT  testing  that  could  be  implemented  and  the  Program 
Manager  should  have  provided  contingency  plans. 

The  Program  Manager  must  balance  all  of  these  factors  to  optimize  the  use  of 
the  validation  tools'  capabilities.  The  survivability  protocols  must  provide  the 
Program  Manager  the  flexibility  to  tailor  his  program  to  meet  the  desired  goals. 


» 
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SECTION  5 


DEVIATIONS  PRECLUDING  ADHERENCE  TO  VALIDATED  PROTOCOLS 

5.1  INTRODUCTION. 

During  the  life-cycle  of  a  program,  situations  can  arise  that  preclude  the  adher¬ 
ence  to  the  survivability  validation  regimen  or  protocols  and  may  require  deviations 
to  the  original  protocols.  The  approved  deviations  must  be  incorporated  into  the 
TEMP  for  proper  disposition.  Deviations  can  be  the  result  of  several  factors,  some  of 
which  are  listed  below. 

5.2  POLITICAL  DECISION  IMPACT  ON  SYSTEM  DEVELOPMENT. 

The  world  situation  may  dictate  that  the  system  under  procurement  be  expe¬ 
dited  as  a  result  of  tense  circumstances.  In  contrast,  diminishing  threat  could  cause 
the  redirection  of  defense  funding  to  social  programs.  This  would  necessitate  an 
evaluation  of  ail  applicable  programs  to  stretch  their  schedules  or  to  be  considered 
for  cancellation. 

5.3  WAIVERS  TO  PROTOCOLS. 

During  the  system  development  cycle,  waivers  to  the  proposed  protocols  may 
need  to  be  granted  due  to  various  circumstances.  The  Program  Management  Team 
can  recommend  that  further  analysis  or  testing  be  waived  and  deviations  made  to 
the  original  protocols  for  many  reasons.  Two  reasons  are:  sufficient  survivability 
design  margins  shown  by  analysis  or  simulation  during  the  early  phases  of  the 
development  cycle  and  comparisons  between  present  system  components  to  already 
tested  and  fielded  components,  which  indicate  that  the  survivability  marqins  are 
high.  Before  these  deviations  can  be  granted,  it  must  be  shown  that  the  hardness 
confidence  limits  are  surpassed.  In  most  cases  these  options  will  be  part  of  the 
protocols,  but  they  are  listed  here  for  completeness. 

Although  the  protocols  should  have  contingencies  to  handle  most  situations,  ' 

the  ones  most  likely  to  not  be  adequatelv  covered  are  changes  in  the  funding  and 
schedule.  As  these  vary,  various  trade-ons  need  to  be  performed  which  may  result  in 
waivers  to  the  protocols.  The  fluctuations  can  come  about  due  to  the  unavailability 
of  proper  validation  tools,  an  acceleration  of  the  program,  a  de-emphasis  of  the 
program  resulting  in  a  stretching  of  the  schedule  or  elimination  of  requirements, 
overruns  in  other  portions  of  the  program  causing  shortfalls  for  survivability  valida-  » 

tion,  etc.  The  magnitude  of  these  changes  cannot  be  anticipated  in  advance,  so 
waivers  to  the  original  protocols  will  be  required  after  the  proper  procedures  to 
handle  the  situation  have  been  determined  and  associated  reviews/approvals  made 
by  the  acquisition  authority. 

5.4  CHANGE  IN  THREAT  SCENARIO.  » 

The  STAR  details  the  accepted  threats  for  the  program.  The  STAR  is  established 
early  in  the  program  cycle  and  is  refined  as  required.  However,  if  a  new  threat  is 
uncovered  due  to  the  development  of  a  new  technology  or  different  use  of  an  old 
technology,  the  system  development  cycle  will  be  impacted.  An  assessment  of 
available  technologies  must  be  made  to  determine  if  a  new  technology  develop-  • 

ment  program  is  required  to  perform  the  survivability  validation  process.  Any  new 
technologies  that  are  required  must  be  identified  as  soon  as  possible  to  assure  their 
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timely  implementation.  If  the  changes  in  the  threats  are  identified  early  in  the 
development  cycle,  adjustments  to  the  program  can  be  made  much  easier  than 
when  the  system  is  nearing  production.  In  that  instance,  a  major  modification  or 
preplanned  product  improvement  program  may  be  required.  In  either  case,  addi¬ 
tional  protocols  must  be  added  to  the  survivability  regimen  to  provide  adequate 
hardness  assurance. 

5.5  ACQUISITION  AUTHORITY  REDIRECTION. 

Protocols  can  be  impacted  by  changes  in  direction  by  the  acquisition  authority. 
These  changes  cannot  be  anticipated  in  the  original  protocols  and  must  be  handled 
on  a  case  by  case  basis  by  modifying  old  protocols  or  initiating  the  development  of 
new  ones.  In  most  instances,  the  acquisition  authority  redirection  is  a  result  of 
Congressional  pressure  or  a  change  in  the  acquisition  strategy.  Any  of  these  will 
invariably  impact  the  schedule  and  resources  required  for  the  procurement, 
generally  with  increases  in  both. 

5.6  DIFFICULTY  OF  PROTOCOL  PROCEDURE  ACCEPTANCE. 

Some  agencies  may  have  a  reluctance  to  use  the  established  protocol  proce¬ 
dures.  This  inertia  may  result  in  an  adherence  to  old  inefficient  methods  which  can 
cause  unnecessary  stress  on  the  program  funds  and  schedule.  Potentially,  greater 
efficiency  could  be  obtained  by  using  a  survivability  regimen  and  its  associated 
protocols,  which  consist  of  validated  hardening  techniques.  Thus  the  agencies  and 
Program  Managers  must  be  convinced  that  using  this  management  technique  will  be 
beneficial.  Acceptance  of  the  protocols  can  be  assisted  by  using  validated  hardening 
techniques  and  making  the  protocols  general  enough  for  use  by  ail  agencies,  but  be 
versatile  enough  to  handle  any  special  requirements  posed  by  tne  operational  envi¬ 
ronment.  For  example,  the  protocol  for  the  nuclear  hardening  of  a  missile  must  be 
general  enough  to  cover  missiles  developed  by  the  three  Services.  However,  it  must 
contain  sections  specific  enough  to  handle  any  special  operational  requirements 
imposed  on  each  Service's  missile. 

5.7  SCHEDULE  SLIPS. 

Changes  to  the  original  protocol  resulting  from  any  of  the  deviations  mentioned 
previously  can  cause  a  schedule  slip  if  they  cannot  be  enacted  quickly  enough  in  the 
acquisition  cycle.  This  will  result  in  reviews  and  inquiries  being  held  at  the  appro¬ 
priate  level.  Depending  upon  the  importance  of  the  project,  these  reviews  can  be 
held  by  the  acquisition  authority,  or  if  a  milestone  is  missed,  reviews  can  be  held  at 
the  Congressional  level.  Not  meeting  the  schedule  can  cause  major  impacts  to  the 
program  such  as  major  cost  overruns  and  delays  in  fielding  the  system.  In  the 
extreme  of  missing  a  milestone,  the  resulting  investigations  could  result  in  a  major 
cutback  of  the  system  or  cancellation  of  the  program.  Thus,  the  regimen  and 
protocols  must  be  flexible  enough  to  permit  rapid  redirection  of  the  effort  to  satisfy 
the  new  survivability  requirements. 


SECTION  6 


PROTOCOL  ADVANTAGES 

The  use  of  regimens  and  protocols  is  a  proactive  management  approach  for 
survivability  validation.  The  previous  discussions  have  primarily  pointed  out  some  of 
the  difficulties  involved  in  protocol  development.  This  section  will  address  some  of 
the  advantages  of  using  protocols. 

As  mentioned  in  earlier  sections,  the  Program  Manager  is  responsible  for  the 
survivability  of  the  system  during  its  entire  life  cycle.  The  regimen  and  protocols  are 
a  valuable  management  approach  to  help  the  Program  Manager  achieve  surviv¬ 
ability  validation  in  an  efficient  manner.  Although  protocols  are  presently  used  in 
various  segments  of  the  acquisition  cycle,  the  proposed  management  method  would 
use  a  regimen  and  protocols  throughout  the  program.  This  departure  from  previous 
management  methods  permits  greater  flexibility  in  establishing  a  survivability 
validation  program.  The  survivability  program  is  tailored  to  the  acouisition  schedule 
and  allotted  resources.  The  selection  of  protocols  is  performed  by  the  Program 
Management  staff.  The  staff  will  more  readily  accept  the  procedures  and  be 
committed  to  the  project  because  they  have  been  intimately  involved  in  developing 
the  survivability  program.  Using  the  protocols  permits  effective  cost/schedule  trade¬ 
offs  to  be  made  during  the  planning  phases  to  help  develop  an  optimum  strategy  for 
the  survivability  program  and  provides  documentation  of  the  survivability  validation 
process. 

Since  the  protocols  are  developed  for  general  use,  the  various  DoD  agencies  can 
choose  the  protocols  that  best  suit  their  purposes.  The  protocols  will  be  documented 
in  a  database  and  be  readily  accessible.  In  addition,  there  will  be  many  protocols 
with  varying  hardness  confidence  leveb  to  permit  efficient  use  of  the  available 
resources.  This  management  procedure  will  reduce  the  costly  duplication  of 
methodologies  by  each  DoD  agency.  The  protocols  can  have  sections  for  the  specific 
needs  of  a  DoD  agency.  Because  or  this  modularity,  the  protocols  can  be  used  by 
many  different  programs  without  further  protocol  development  expense.  For 
instance,  the  protocol  for  a  UGT  for  a  communications  subsystem  could  have  sections 
of  the  protocol  v/hich  would  describe  the  specific  hardness  validation  procedures  for 
land,  sea,  air,  or  space  systems.  This  would  permit  the  various  DoD  agencies  to  use 
the  same  basic  protocol  for  any  communication  system,  but  the  specific  implementa¬ 
tion  for  an  operational  environment  (land,  sea,  air,  or  space)  would  be  located  in 
different  branches  of  the  protocol.  The  basic  protocol  format  would  capitalize  on 
the  similarities  of  protocols  already  developed  by  the  various  agencies. 

To  permit  a  more  versatile  survivability  program,  the  protocols  will  be  written  to 
allow  deviations  in  the  implementation  of  the  regimen  and  protocols  without  exten¬ 
sive  disruption  to  the  survivability  program.  Since  the  regimen  and  protocols  are 
tailored  for  each  phase  of  the  development  cycle  and  updated  to  reflect  develop¬ 
ments  in  the  acquisition,  there  is  a  smooth  transition  between  the  acquisition 
phases.  Because  the  protocols  will  already  be  validated,  the  various  review  boards 
can  readily  determine  if  the  selected  procedures  are  adequate  to  perform  the 
survivability  validation.  Approval  cycles  are  made  2asier,  since  the  entire  surviv¬ 
ability  program  life-cycle  is  available  for  review,  with  the  major  detail  centered  on 
the  development  phase  under  review.  The  subsequent  reviews  are  expedited 
because  the  basic  Program  Plan  has  already  been  described  at  the  previous  reviews 
and  only  changes  or  further  detail  would  be  added. 
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Early  approval  of  the  protocols  using  the  established  procedures  and  criteria 
provide  many  advantages  to  the  Program  Office  and  acquisition  authority,  because 
each  has  a  good  idea  of  the  survivability  implementation  plan  well  in  advance  of  its 
implementation.  This  procedure  optimizes  feedback  to  the  acquisition  authority 
regarding  deficiencies  in  technologies  or  protocols  to  speed  up  any  development 
programs  that  are  required.  It  also  permits  Program  Office  personnel  to  plan  for 
long  lead  items  in  the  development  cycle. 
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SECTION  7 


ISSUES 


7.1  INTRODUCTION. 

The  previous  sections  have  addressed  many  of  the  factors  involved  in  developing 
and  using  protocols.  It  has  been  shown  that  the  use  of  this  management  technique 
will  permit  greater  efficiency  in  the  survivability  validation  process  of  a  system.  In 
addition,  using  protocols  will  provide  extensive  savings  of  time  and  resource's  during 
the  program  life-cycle.  This  is  due  primarily  to  the  use  of  pre-approved  systematic 
methods  to  obtain  the  prescribed  hardness  confidence  levels  required  for  the  system 
dep  oyment. 


7.2  SURVIVABILITY  ISSUES. 


There  are  several  survivability  issues  that  must  be  considered  during  the  life- 
cycle  of  a  system.  Some  of  these  are  presented  in  Table  7-1 ,  which  provides  a  list  of 
nuclear  survivability  actions  required  during  the  life  cycle  of  a  system.  These  issues 
must  be  sufficiently  addressed  by  the  regimen  and  protocols  selected  for  the  surviv¬ 
ability  validation  program.  An  effective  management  procedure  must  address  each 
of  these  issues.  As  a  result  ,the  regimen  and  its  associated  protocols  would  be  used 
at  the  onset  of  the  program  and  become  more  detailed  and  comprehensive  with 
each  milestone.  The  associated  protocols  for  each  regimen  will  have  established 
procedures,  tests,  or  analyses  for  each  system  element  requiring  survivability 
validation. 


Protocols  must  account  for  the  levels  of  system  integration  from  piece-part  to 
system-of-systems.  Different  validation  protocols  may  be  required  at  each  level. 
Table  7-2  provides  the  hierarchy  of  level  of  integration  of  a  syctem-of-systems,  which 
is  defined  as  the  highest  level  of  integration.  Figure  7-1  depicts  how  the  protocols 
for  the  system  utilize  the  tools  and  procedures  appropriate  for  the  hierarchy  level. 
The  results  of  the  protocols  from  piece-parts  to  the  system  are  integrated  to  provide 
the  required  level  of  survivability  for  the  system. 

The  protocols  must  sufficiently  address  hardness  assurance,  hardness,  main¬ 
tenance,  and  hardness  surveillance  to  have  a  successful  survivability  program.  These 
programs  are  designed  to  ensure  that  the  hardened  design  is  produced  correctly, 
assure  that  the  hardened  features  are  maintained  to  design  specifications,  and  make 
sure  that  the  hardening  features  are  monitored  to  determine  if  they  have  degraded. 
The  protocols  must  show  that  the  following  three  elements  have  been  adequately 
addressed  in  the  survivability  validation  program: 

(1)  Hardness  assurance  assures  that  the  hardness  features  are  produced  as 
designed; 

(2)  Hardness  maintenance  provides  instructions  to  preserve  the  hardened 
design  during  operations,  maintenance,  and  logistics  or  support  activities; 

(3)  Hardness  surveillance  examines  the  fielded  equipment  to  determine 
whether  the  hardening  features  of  the  system  have  been  degraded.  In  addition,  an 
assessment  is  made  of  the  hardness  maintenance  program  to  determine  if  it  is 
adequate  for  maintaining  the  system. 
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Table  7-1 .  Survivability  actions  during  system  life  cycle. 


MILESTONE  DECISION  POINT 


SURVIVABILITY  ACTIONS 


Concept  Studies  Approval 


Concept  Demonstration  Approval 


Development  Approval 


Production  Approval 


Major  Modification  Approval 


Establish  survivability  requirements 

Verify  survivability  requirements 
Establish  survivability  criteria  and 
identify  in  Operational  Requirements 
Document 

Include  key  survivability  objectives  in 
Concept  Baseline 

Test  and  Evaluation  criteria  included  in 
Test  and  Evaluation  Master  Plan 
(TEMP). 

Identify  initial  survivability  technology 
shortfalls 

Facilities  characteristics  tracked 
through  the  Integrated  Logistics 
Suppoit  Plan  (ILSP) 

Identify  critical  items  requiring  tests 
and  include  in  TEMP 
Include  survivability  objectives  in  the 
Development  Baseline,  System 
Specification,  and  ILSP 
Develop  hardness  assurance, 
maintenance  and  surveillance 
programs  and  include  in  ILSP 
Address  survivability  issues  in 
Integrated  Program  Summary 

Assess  how  well  survivability  objectives 
have  been  met  and  include  results  in 
initial  Production  Report 
Resolve  any  survivability  issues 
Include  survivability  objectives  in 
Production  Baseline 
Complete  development  and 
implement  hardness  assurance 
program  and  HM/HS  plans 

Review  engineering  change  proposals 

for  survivability  issues 

Support  hardness  assurance  program 
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Table  7-2.  Hierarchy  of  levels  of  integration  of  a  system-of-systems. 


Level  of  integration 
(Highest  to  Lowest) 

Examples 

System-of-Systems  (SOS) 

GPALS,  Trident,  Theater  Missile 

Defense 

System  element  (SE)  level 

Constellations  of  a  particular  satellite, 
e.g..  Brilliant  Pebbles 

System  element  platform  (SEP)  level 

A  radar,  satellite,  an  individual  Brilliant 
Pebble,  a  missile 

Subsystem  level 

Power  subsystem  of  a  satellite,  seeker 
of  a  kinetic  kill  vehicle,  post-boost 
vehicle  guidance  system 

Component  level 

Individual  electronics  boxes,  lenses, 
mirrors 

Part/material  level 

Baffle  materials,  piece  parts 

The  regimen  of  protocols  is  not  intended  to  be  used  as  an  inflexible  set  of 
procedures  to  be  implemented  for  all  systems.  Instead,  the  regimen  is  tailored  for  a 
specific  procurement  and  becomes  a  living  management  tool  that  is  enhanced  as  the 
acquisition  proceeds.  In  addition,  the  protocols  cover  a  myriad  of  analysis  methods, 
simulations,  and  tests  that  can  be  selected  to  validate  the  survivability  of  the  system. 
Each  protocol  has  a  certain  level  of  risk  and  confidence  associated  with  its  use.  This 
permits  the  Program  Manager  flexibility  in  selecting  the  optimum  set  of  protocols  to 
achieve  the  desired  level  of  survivability  confidence  commensurate  with  the  mission 
requirements,  program  schedule,  and  financial  constraints. 

A  "proof-of-hardness"  cannot  be  obtained  without  subjecting  the  system  to  the 
true  threat  environment  and  assessing  the  results  of  that  environment  on  the 
system.  For  many  environments  actual  testing  is  not  practical,  as  is  the  case  for 
assessing  nuclear  effects  on  large  distributed  systems.  Thus,  test  and  analysis  play 
complementary  roles  in  validating  system  hardness.  Comparison  of  test  data  with 
analytical  predictions  provides  confidence  in  the  analytical  approach  and  under¬ 
standing  of  the  system.  In  turn,  the  analysis  can  be  er.trapolated  to  the  threat  envi¬ 
ronment  with  increased  confidence.  The  validation  process  is  then  an  iterative 
process  in  which  the  principles  of  good  engineering  practice,  the  results  of  analytical 
calculations,  simulations,  and  test  data  are  used  to  assess  system  hardness  with  the 
level  of  confidence  dependent  on  the  methods  used. 

Protocol  selection  permits  trade-offs  between  analysis  and  testing.  During  the 
initial  design  phases,  analysis  will  have  a  primary  role  with  support  from  empirical 
data.  As  the  development  proceeds  from  the  design  phase  to  hardness  validation 
and  operation,  testing  will  assume  a  more  dominant  role  with  support  from  analysis. 
However,  in  many  cases  (as  in  system-of-systems  and  ground-based  facilities)  a  full 
test  simulation  is  not  possible,  nor  is  a  complete  analytical  evaluation  possible  due  to 
the  vast  complexity  of  many  systems.  Hence,  throughout  all  phases  of  the  system's 
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System  Protocol 


Assured  Survivability 
Figure  7-1 .  Systems  survivability  validation  process. 


22 


life-cycle,  a  balance  between  testing  and  analysis  is  used  to  evaluate  the  surviv¬ 
ability. 

Care  must  be  exercised  when  implementing  hardening  measures  because  the  _ 

hardening  for  one  threat  can  impact  the  effectiveness  of  other  related  threats.  One 
example  is  the  electromagnetic  (EM)  disciplines  of  electromagnetic  compatibility, 
electromagnetic  interference  control,  lightning  protection,  TEMPEST  control,  and 
electromagnetic  pulse  (EMP)  protection.  In  some  cases,  protection  against  one 
effect  may  also  provide  protection  against  another,  but  often  conflicts  may  arise 
between  the  protection  practices  utilized  for  these  EM  effects.  The  appropriate 
protocol  should  indicate  this  interrelationship.  For  example,  it  is  already  recognized  • 

within  the  DoD  that  it  is  advantageous  to  combine  all  the  electromagnetic  effects 
into  a  few  standards  and  requirements.  Thus,  the  protocols  utilized  for  EMP  pro¬ 
tection  can  alert  the  user  to  the  practices  used  for  protection  from  the  other  EM 
effects.  However,  the  EMP  protocol  does  not  have  to  incorporate  these  practices 
from  the  other  EM  disciplines.  This  encourages  the  development  of  a  good  system 
design  which  incorporates  integrated  electromagnetic  effects  protection  • 

techniques,  whenever  practical. 

7.3  TRADEOFF  ISSUES. 

The  use  of  regimens  and  protocols  will  help  the  Program  Manager  and  his  staff 
perform  quantitative  tradeoffs  to  assist  in  the  planning  stages  and  updating  of  a  » 

system's  survivability  validation  program  throughout  its  life  cycle.  Table  7-3  lists 
several  of  the  cost  trade-off  factors  that  can  be  considered.  The  results  of  these 
analyses  permit  flexibility  in  selecting  the  optimum  set  of  protocols  to  achieve  the 
desired  level  of  survivability  commensurate  with  the  mission  requirements,  program 
schedule,  and  financial  constraints. 

» 

During  the  course  of  the  protocol  pilot  study,  example  quantifications  of  these, 
factors  for  a  specific  scenario  will  be  performed  to  describe  to  the  community  the 
cost  and  time  issues  associated  with  protocols.  In  addition,  a  basic  quantification  of 
the  resources  required  to  create,  update,  maintain,  and  establish  configuration 
control  of  the  protocol  database  should  be  performed.  This  will  provicTe  decision 
makers  with  estimates  of  the  resources  required  for  implementing  the  protocol  P 

management  technique. 

Table  7-3.  Potential  protocol  trade-off  factors. 


Confidence  limits 

Detail/fidelity  of  protocol 

Protocol  updates  (for  modified  or  new  systems) 

Implementation  factors 

Availability/accessibility  of  tools 
Utilization  costs 
Validation  tool  improvements 
Waivers 
Schedule 
Cost  analyses 
Application  costs 


i 


» 


» 
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7.4  OTHER  ISSUES. 

There  are  several  outstanding  issues  that  will  be  considered  during  the  develop¬ 
ment  of  the  protocols.  These  are  listed  below: 

(1)  An  adequate  list  of  commercial  and  government  test  facilities, 
standards,  handbooks,  data  items,  and  available  analysis/simulation  codes  must  be 
updated  and  configuration  control  must  be  imposed  and  maintained  for  as  long  as 
the  database  is  utilized.  This  will  be  the  basis  set  for  the  protocols  and  will  be  used 
by  Program  Office  personnel  in  developing  the  survivability  regimen.  The  basis  set 
will  consist  of  only  validated  protocols.  Thus,  community  acceptance  of  the  valida¬ 
tion  techniques  will  be  easier,  because  the  protocols  are  already  in  general  use; 

(2)  A  determination  must  be  made  as  to  the  level  of  generality  of  the 
protocols  required  to  make  a  viable  set  of  protocols  for  all  types  of  situations. 
Protocols  from  each  of  the  DoD  agencies  should  be  used  and  various  techniques 
specific  to  a  particular  agency  must  be  incorporated.  This  will  also  assist  in  accept¬ 
ance  of  the  protocols; 

(3)  The  resistance  of  agencies  to  adopt  a  new  management  technique  must 
be  addressed.  This  can  be  accomplished  by  informing  the  community  through 
conference  papers,  technical  meetings,  and  reports  addressing  the  advantages  of 
using  protocols.  Support  for  the  new  management  technique  can  be  obtained  by 
inviting  the  interested  agencies  to  workshops  to  assist  in  developing  some  of  the 
protocols.  The  protocol  introduction  process  must  show  the  Program  Managers  that 
they  will  be  provided  with  enough  information  to  perform  efficient  cost/confidence 
trade-offs  during  protocol  seleaion.  In  addition,  it  must  be  shown  how  the  proto¬ 
cols  will  help  the  Program  Office  with  the  hardness  assurance,  hardness  validation, 
and  hardness  maintenance/hardness  surveillance  (HM/HS). 

(4)  Investigations  are  required  as  to  the  best  methods  available  to  validate 
the  survivability  of  a  system  that  can't  be  tested,  such  as  a  system  that  has  widely 
separated  components.  Once  the  techniques  have  been  established,  the  protocols 
can  be  developed. 
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List  of  Abbreviations 


AGT 

Aboveground  Test 

CTP 

Critical  Technical  Parameters 

DoD 

Department  of  Defense 

EM 

EMP 

Electromagnetic 

Electromagnetic  Pulse 

GPALS  - 

Global  Protection  Against  Limited  Strikes 

HM/HS  - 

Hardness  Maintenance/Hardness  Surveillance 

ILSP 

Integrated  Logistics  Support  Plan 

MAOPR  - 

Minimum  Acceptable  Operational  Performance 
Requirements 

SE 

SEP 

SOS 

STAR  - 

System  Element 

System  Element  Platform 

System-of-Systems 

System  Threat  Assessment  Report 

TEMP  - 

Test  and  Evaluation  Master  Plan 

UGT 

U.S. 

Underground  Test 

UnitecTstates 
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